Transmitting method, receiving method, transmitting device, and receiving device

ABSTRACT

A transmitting method according to one aspect of the present disclosure includes transmitting a first stream, the first stream including: timing update identification information id 1  indicating whether or not a correspondence relationship between a first reference clock and a second reference clock has been updated, the first reference clock being used to transmit and receive the first stream, and the second reference clock being used to transmit and receive a second stream related to another content to be reproduced in synchronization with the content related to the first stream; a first time according to the first reference clock; and a second time according to the second reference clock, the second time being associated with the first time based on the updated correspondence relationship.

BACKGROUND

1. Technical Field

The present disclosure relates to a transmitting method and a receivingmethod of data.

2. Description of the Related Art

Currently, standardization related to a data structure of auxiliary dataindicating a relationship between PTSs (Presentation Time Stamp) ofaccess units of streams which are independently transmitted isprogressing (see, for example, NPL 1). The streams which areindependently transmitted include streams transmitted according toMPEG-2 TS (Moving Picture Experts Group-2 Transport Stream), and streamstransmitted in multiplexing formats such as MPEG-DASH (Dynamic AdaptiveStreaming over HTTP) or MMT (MPEG Media Transport). Associating suchPTSs of access units will be hereinafter referred to as timelineextension.

CITATION LIST Patent Literature

-   NPL 1: Working Draft of ISO/IEC13181-1:2012/AMD6-Delivery of    Timeline for External Data

SUMMARY

In one general aspect, the techniques disclosed here feature atransmitting method for transmitting a first stream related to contentof an image or audio, the method comprising transmitting the firststream, the first stream including: timing update identificationinformation indicating whether or not a correspondence relationshipbetween a first reference clock and a second reference clock has beenupdated, the first reference clock being used to transmit and receivethe first stream, and the second reference clock being used to transmitand receive a second stream related to another content to be reproducedin synchronization with the content related to the first stream; a firsttime according to the first reference clock, and a second time accordingto the second reference clock, the second time being associated with thefirst time based on the updated correspondence relationship.

It should be noted that general or specific embodiments may beimplemented as a system, a method, an integrated circuit, a computerprogram, a storage medium, or any selective combination thereof.

As a transmitting method and a receiving method according to the presentdisclosure, there is provided a transmitting method and a receivingmethod in which timeline extension can be efficiently performed.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a view illustrating a syntax of a TEMI access unit fordescribing a conventional problem;

FIG. 1B is a flowchart illustrating processing in which a receivingdevice synchronizes a reference source timeline and a referencedestination timeline by using the TEMI access unit;

FIG. 1C is a view illustrating a relationship between time T_org andtime T_ref in the reference source timeline and the referencedestination timeline;

FIG. 2 is a view illustrating an example of a syntax of a TEMI accessunit according to an exemplary embodiment;

FIG. 3 is a block diagram illustrating a configuration of a transmittingdevice according to the exemplary embodiment;

FIG. 4 is a flowchart illustrating an example of a processing operationof the transmitting device according to the exemplary embodiment;

FIG. 5 is a block diagram illustrating a configuration of a receivingdevice according to the exemplary embodiment;

FIG. 6 is a flowchart illustrating an example of a processing operationof the receiving device according to the exemplary embodiment;

FIG. 7A is a view illustrating a syntax of a TEMI access unit accordingto a second modified example of the exemplary embodiment;

FIG. 7B is a view illustrating a syntax of a TEMI access unit accordingto the second modified example of the exemplary embodiment;

FIG. 7C is a view illustrating a syntax of a TEMI access unit accordingto the second modified example of the exemplary embodiment;

FIG. 8A is a view for describing a random access performed by areceiving device according to the second modified example of theexemplary embodiment;

FIG. 8B is a view for describing processing operations of a transmittingdevice and the receiving device when a time lag is taken into accountaccording to the second modified example of the exemplary embodiment;

FIG. 9 is a flowchart illustrating an example of the processingoperation of the receiving device according to the second modifiedexample of the exemplary embodiment;

FIG. 10A is a flowchart illustrating a transmitting method according toone aspect of the present disclosure;

FIG. 10B is a block diagram of the transmitting device according to oneaspect of the present disclosure;

FIG. 11A is a flowchart illustrating a receiving method according to oneaspect of the present disclosure; and

FIG. 11B is a block diagram of a receiving device according to oneaspect of the present disclosure.

DETAILED DESCRIPTION

(Underlying Knowledge Forming Basis of the Present Disclosure)

The inventors of the present disclosure found that a transmitting methodand a receiving method according to NPL 1 described in the section ofBackground Art had following problems.

According to conventional timeline extension, auxiliary data streams aredefined. In this case, streams which are transmitted independently fromeach other will be referred to as a reference source and a referencedestination. Each of these streams includes items of different contentindicating an image or audio. Further, the reference source is anauxiliary data stream. In one of PES (Packetized Elementary Stream)packets included in this auxiliary data stream, a TEMI (Timeline andExternal Media Information) access unit is stored. Further, in a headerof this PES packet, time T_org of a reference source timeline isindicated. Furthermore, in the TEMI access unit, time T_ref which is atime corresponding to time T_org in the reference destination timelineis stored. A timeline is a reference clock used to transmit and receivestreams. That is, on a common time axis, time T_org at the referencesource and time T_ref at the reference destination are mutually equaltimes.

The TEMI access unit is transmitted with a reference source stream. Areceiving device can synchronize and reproduce reference source contentand reference destination content by analyzing the TEMI access unit.

FIG. 1A is a view illustrating a syntax of the TEMI access unit fordescribing a conventional problem.

As illustrated in FIG. 1A, the TEMI access unit (TEMI_AU) includesaddon_location and timescale.

addon_location indicates reference destination location information(e.g., URL: Uniform Resource Locator).

timescale includes time_before_activation or media_time_anchor.

time_before_activation indicates a time required until addon_locationbecomes effective (i.e., becomes possible to obtain a referencedestination).

media_time_anchor indicates time T_ref of the reference destinationtimeline.

Further, time T_org of the reference source timeline is indicated in aPTS field of a header of the PES packet in which the TEMI access unit isstored.

FIG. 1B is a flowchart illustrating processing in which the receivingdevice synchronizes a reference source timeline and a referencedestination timeline by using the TEMI access unit.

First, the receiving device receives data included in the referencesource stream (step S1001). Next, the receiving device determineswhether or not the received data is the TEMI access unit, i.e., whetheror not the TEMI access unit has been received (step S1002). When it isdetermined that the TEMI access unit has not been received (“NO” in stepS1002), the receiving device repeatedly executes processing from stepS1001. On the other hand, when it is determined that the TEMI accessunit has been received (“YES” in step S1002), the receiving deviceobtains a PTS of the PES packet in which the TEMI access unit is stored,and media_time_anchor in the TEMI access unit. That is, the receivingdevice obtains time T_org and time T_ref (step S1003). Next, thereceiving device synchronizes the reference source timeline and thereference destination timeline based on time T_org and time T_ref (stepS1004). Note that, synchronizing the reference source timeline and thereference destination timeline is to specify a correspondencerelationship between these timelines, i.e., a time offset (i.e., offsetvalue).

FIG. 10 is a view illustrating a relationship between time T_org andtime T_ref in the reference source timeline and the referencedestination timeline.

The receiving device specifies a difference between time T_org and timeT_ref as the time offset (offset). This time offset is expressed asoffset=T_ref−T_org×(timescale 2/timescale 1). Here, timescale 1 is atimescale of the reference source timeline, and timescale 2 is atimescale of the reference destination timeline. Note that, eachtimescale is a frequency which expresses a time and, more specifically,a number of times which is expressed per unit time (e.g. one second).When specifying such a time offset, the receiving device can calculate atime of the reference destination timeline corresponding to arbitrarytime T of the reference source timeline according to T+offset.

However, this timeline extension has a problem that, every time a TEMIaccess unit is received, time T_org and time T_ref need to be analyzed,a processing amount related to management of time information uponsynchronization and reproduction is large and processing is notefficient.

To solve this problem, a transmitting method according to one aspect ofthe present disclosure is a transmitting method for transmitting a firststream related to content of an image or audio, and the method includingtransmitting the first stream including timing update identificationinformation indicating whether or not a correspondence relationshipbetween a first reference clock and a second reference clock has beenupdated, the first reference clock being used to transmit and receivethe first stream, and the second reference clock being used to transmitand receive a second stream related to another content to be reproducedin synchronization with content related to the first stream, a firsttime of the first reference clock, and a second time of the secondreference clock, the second time being associated with the first timebased on the updated correspondence relationship.

Thus, the timing update identification information is transmitted to thereceiving device. Consequently, the receiving device can easilydetermine whether or not the correspondence relationship between thefirst reference clock and the second reference clock has been updatedbased only on this timing update identification information withoutmaking determination by performing analysis using the first time and thesecond time. As a result, only when it is determined that thecorrespondence relationship has been updated, the receiving device needsto specify the updated correspondence relationship by using the firsttime and the second time and, consequently, a processing amount of thereceiving device can be suppressed. Consequently, it is possible toefficiently perform timeline extension.

For example, the transmitting method may further include determiningwhether or not the correspondence relationship has been updated, andgenerating the timing update identification information indicating aresult of the determination.

Further, the determination of whether or not the correspondencerelationship has been updated may include determining whether or not adiscontinuous change is present in a change in a time indicated by thefirst reference clock, and determining, when it is determined that thediscontinuous change is present, that the correspondence relationshiphas been updated.

Furthermore, the first reference clock may be a PCR (Program ClockReference) according to MPEG2-TS (Moving Picture Experts Group-2Transport Stream).

Still further, the transmitting method may include, when it isdetermined that the correspondence relationship has been updated:calculating an offset value corresponding to the updated correspondencerelationship between the first reference clock and the second referenceclock; and including, in the first stream, the first time and the secondtime that is specified based on the calculated offset value.

Moreover, the timing update identification information and the secondtime may be stored in a TEMI (Timeline and External Media Information)access unit of the first stream.

Moreover, the first time may be stored in a PES (Packetized ElementaryStream) packet in which the TEMI access unit is stored.

Further, the TEMI access unit may include a timeline descriptor and alocation descriptor, and the transmitting method may include: storingthe timing update identification information and the second time in thetimeline descriptor; and storing location information in the locationdescriptor, the location information indicating a location of thecontent related to the second stream.

Furthermore, the transmitting method may further include: transmittinganother location descriptor after the location descriptor istransmitted, the another location descriptor including splicingidentification information and location information, the splicingidentification information being used to stop reproduction of thecontent related to the second stream and start reproduction of contentrelated to a third stream, and the location information indicating alocation of the content related to the third stream; and retransmittingthe location descriptor by a time when a reproduction period of thecontent related to the third stream ends.

Thus, another location descriptor is transmitted to the receivingdevice. Consequently, the receiving device can stop reproduction ofcontent related to the second stream based on the splicingidentification information stored in the another location descriptor,and reproduce the content related to the third stream. Further, thelocation descriptor in which the location information indicating alocation of the content related to the second stream is stored isretransmitted to the receiving device by the time when a reproductionperiod (i.e., splice interval) of the content related to this thirdstream ends. Consequently, even when reception of the first stream isstarted in this reproduction period, at the end of this reproductionperiod, the receiving device can easily specify the content to bereproduced next and reproduce the content. That is, similar to a casewhere reception of the first stream is started before the reproductionperiod, the receiving device can resume reproduction of the contentrelated to the source second stream, the content which has beenreproduced before the content related to the third stream.

Further, a receiving method according to one aspect of the presentdisclosure is a receiving method for receiving a first stream related tocontent of an image or audio, the receiving method including receivingthe first stream including timing update identification informationindicating whether or not a correspondence relationship between a firstreference clock and a second reference clock has been updated, the firstreference clock being used to transmit and receive the first stream, andthe second reference clock being used to transmit and receive a secondstream related to another content to be reproduced in synchronizationwith content related to the first stream, a first time of the firstreference clock, and a second time of the second reference clock, thesecond time being associated with the first time based on the updatedcorrespondence relationship.

Thus, the timing update identification information is received by thereceiving device. Consequently, the receiving device can easilydetermine whether or not the correspondence relationship between thefirst reference clock and the second reference clock has been updatedbased only on this timing update identification information withoutdeliberately making determination by performing analysis using the firsttime and the second time. As a result, only when it is determined thatthe correspondence relationship has been updated, the receiving deviceneeds to specify the updated correspondence relationship by using thefirst time and the second time and, consequently, a processing amountcan be suppressed. Consequently, it is possible to efficiently performtimeline extension.

Note that these comprehensive or specific aspects may be realized by asystem, a method, an integrated circuit, a computer program, or arecording medium such as a computer-readable CD-ROM, or may be realizedby an arbitrary combination of systems, methods, integrated circuits,computer programs, or recording media.

Hereinafter, the exemplary embodiment will be more specificallydescribed with reference to the drawings.

In addition, the exemplary embodiment described below indicates acomprehensive or specific example. Numerical values, shapes, materials,components, disposition positions and a connection mode of thecomponents, steps and an order of the steps are exemplary, and do notintend to limit the present disclosure. Further, the components whichare not recited in an independent claim representing the highest conceptamong the components according to the following exemplary embodimentwill be described as arbitrary components.

Exemplary Embodiment

FIG. 2 is a view illustrating an example of a syntax of a TEMI accessunit according to the present exemplary embodiment.

The TEMI access unit includes addon_location, timescale,media_time_anchor described above, and further includes is_timing_updatewhich is timing update identification information id1.

The is_timing_update (id1) indicates that a correspondence relationship(e.g. time offset) between a reference source timeline and a referencedestination timeline has been updated. That is, is_timing_updateindicates that time T_org and time T_ref are stored in a PES packetafter the correspondence relationship is updated. Note that thereference source timeline and the reference destination timeline will bereferred to as both timelines. Further, media_time_anchor indicates timeT_ref, and time T_org is indicated as a PTS in a header of a PES packetincluding this TEMI access unit.

For example, is_timing_update is a one-bit flag. Hence, in a case ofis_timing_update=1, the is_timing_update indicates that thecorrespondence relationship between the both timelines has been updated.Further, in a case of is_timing_update=0, the is_timing_update indicatesthat the correspondence relationship between the both timelines has notbeen updated.

The correspondence relationship between the both timelines is updated ina period in which addon_location is effective. Hence, when a value ofis_timing_update is set to 1, a value of is_announcement is set to 0 anda field of media_time_anchor is present.

The transmitting device sets is_timing_update to 1 only in the TEMIaccess unit transmitted immediately after the correspondencerelationship between the both timelines is updated. Further, thetransmitting device may set is_timing_update to 1 in a plurality of TEMIaccess units transmitted after the update. Furthermore, in order tonotify the update in advance, the transmitting device may setis_timing_update to 1 in a plurality of TEMI access units before theupdate and in a TEMI access unit immediately after the update. In thiscase, the receiving device may determine that the correspondencerelationship between the both timelines has been updated by determiningthat a value of is_timing_update has changed from 1 to 0.

The transmitting device may set a timescale of the reference destinationtimeline to “timescale” of the TEMI access unit. Thus, the transmittingdevice can use a value of media_time_anchor as time information of thereference destination timeline without converting the timescale.

For example, when a reference source is transmitted according toMPEG2-TS and a reference destination is transmitted according toMPEG-DASH, a TEMI access unit is multiplexed on a TS. Further, thereference source may be transmitted according to MPEG2-TS, the referencedestination may be transmitted as an MMT stream, the reference sourceand the reference destination may be transmitted according to MPEG2-TS,or another combination of the reference source and the referencedestination may be transmitted.

Furthermore, the transmitting device may transmit a TEMI access unit asanother stream without multiplexing the reference source stream with aTEMI access unit.

The transmitting device updates the correspondence relationship betweenthe reference source timeline and the reference destination timeline in,for example, following cases (cases 1 and 2).

That is, the transmitting device updates the correspondence relationshipwhen discontinuity occurs in a reference source reference clock (case1). For example, the transmitting device updates the correspondencerelationship when discontinuity occurs in a PCR (Program ClockReference) according to MPEG2-TS.

Alternatively, the transmitting device updates the correspondencerelationship when discontinuity occurs in a reference destinationreference clock (case 2). For example, the transmitting device updatesthe correspondence relationship when part of MP4 Fragment is skippedaccording to reference destination MPEG-DASH. Alternatively, thetransmitting device updates the correspondence relationship whendiscontinuity occurs in a PCR according to reference destinationMPEG2-TS. Alternatively, the transmitting device updates thecorrespondence relationship when discontinuity occurs in a value of anNTP (Network Time Protocol) which is the reference clock according toreference destination MMT.

In addition, in case 2, occurrence of discontinuity in the referencedestination reference clock and a displacement amount of the referenceclock are notified to the transmitting device which transmits areference source stream.

Next, a method for setting media_time_anchor will be described.

For example, according to MPEG-DASH, data is received from a middlesegment of segmentalized content in some cases. In this case, when a PTSof a head access unit in a display (reproduction) order in each segmentor a period of MPEG-DASH complies with a timeline (zero timeline) whichsets a head of content to zero, it is possible to use this timeline.Meanwhile, when each segment or a PTS complies with a timeline (referredto as a segment timeline) which sets to zero a PTS of a head access unitin a segment or the like which starts being received, media_time_anchoris also set according to the segment timeline.

In this case, an index number or the like for specifying a segment whichserves as a reference of a segment timeline may be stored in adescriptor in a PMT (Program Map Table) in which a TEMI access unit or aPID (packet identifier) of a stream of the TEMI access unit is stored.Note that, when the segment timeline is used, it is necessary to obtainin advance a time of the reference source timeline which matches with atime at which a time of the segment timeline becomes zero.

In a case of a format in which a reference clock is defined such asMPEG2-TS, MMT, or a RTP (Real-time Transport Protocol), a value ofmedia_time_anchor is set according to the reference clock to be used.The reference clock is defined by a PCR according to MPEG2-TS, and thereference clock is defined by an NTP (Network Time Protocol) accordingto MMT and RTP.

FIG. 3 is a block diagram illustrating a configuration of the abovetransmitting device according to the present exemplary embodiment.

Transmitting device 100 according to the present exemplary embodimentincludes reference source TM deciding unit 101, discontinuitydetermining unit 102, identification information setting unit 103,discontinuity setting unit 104, and normal setting unit 106.

Reference source TM determining unit 101 determines time T_org of thereference source. Discontinuity determining unit 102 determines whetheror not discontinuity has occurred in the reference source timeline.Identification information setting unit 103 sets is_timing_update whichis timing update identification information id1 described above. Whenthe correspondence relationship between the both timelines is updated,discontinuity setting unit 104 sets time T_ref of the referencedestination timeline to a TEMI access unit according to the updatedcorrespondence relationship. When the correspondence relationshipbetween the both timelines is not updated, normal setting unit 106 setstime T_ref of the reference destination timeline to a TEMI access unitaccording to the correspondence relationship which has been used so far.

FIG. 4 is a flowchart illustrating an example of a processing operationof transmitting device 100 according to the present exemplaryembodiment.

A case where the reference source stream is transmitted according toMPEG2-TS will be described with reference to FIG. 4.

First, reference source TM deciding unit 101 of transmitting device 100decides time T_org of the reference source timeline, and stores timeT_org in a header of a PES packet in which a TEMI access unit is stored(step S101).

Next, discontinuity determining unit 102 determines whether or notdiscontinuity has occurred in a PCR which is the reference clock afterthe last TEMI access unit is transmitted (step S102). When it isdetermined that the discontinuity has occurred in the PCR (“YES” in stepS102), identification information setting unit 103 sets a flag ofis_timing_update of the TEMI access unit to 1 (step S103). Further,discontinuity setting unit 104 calculates a time offset after thereference clock is switched, and sets time T_ref (media_time_anchor) ofthe reference destination timeline based on this time offset (stepS104).

On the other hand, when it is determined that the discontinuity has notoccurred in the PCR (“NO” in step S102), identification informationsetting unit 103 sets a flag of is_timing_update of the TEMI access unitto 0 (step S105). Further, normal setting unit 106 sets time T_ref(media_time_anchor) of the reference destination timeline based on thetime offset which has been used so far (step S106).

Discontinuity determining unit 102 may directly obtain a PCR from meanswhich generates the PCR, or may analyze a TS packet in which the PCR isstored. In a latter case, when a field (discontinuity_indicator)indicating whether or not discontinuity has occurred in the PCR of theheader of the TS packet is set to 1, discontinuity determining unit 102determines that the discontinuity has occurred. Alternatively, whendiscontinuity_indicator is continuously set to 1 prior to the occurrenceof the discontinuity, a value of the updated PCR is stored in a last TSpacket in which the value of the discontinuity is 1. Accordingly,discontinuity determining unit 102 may determine that the discontinuityhas occurred when this last TS packet is received.

Transmitting device 100 may transmit a TEMI access unit per audio orvideo access unit (hereinafter referred to as an AV access unit)multiplexed on the reference source stream, or may cyclically transmit aTEMI access unit such that a transmission interval is a predeterminedvalue or less.

When the TEMI access unit is transmitted per AV access unit,transmitting device 100 may set a PTS of a header of a PES packet suchthat the PTS of the header of the PES packet in which the TEMI accessunit is stored is equal to a PTS of the AV access unit.

In this case, transmitting device 100 may transmit the TEMI access unittemporally in advance compared to the AV access unit corresponding tothis TEMI access unit. Thus, the receiving device can obtain a time ofthe reference destination timeline corresponding to the PTS of this AVaccess unit before the AV access unit is obtained.

Further, when the TEMI access unit is cyclically transmitted,transmitting device 100 may set a PTS of a PES packet such that the PTSof the PES packet in which the TEMI access unit is stored is equal to aPTS of the AV access unit. Furthermore, in the case where acorrespondence relationship between the reference source timeline andthe reference destination timeline has been updated, transmitting device100 transmits a TEMI access unit corresponding to a PTS of an accessunit immediately after the update without waiting for a timing totransmit a next TEMI access unit.

Transmitting device 100 which transmits the reference source stream mayobtain information of a reference destination timeline from atransmitting device which transmits a reference destination stream byusing means which is not illustrated. For example, when reference sourcecontent includes a main part and CMs (commercials), and referencedestination content includes only a main part which is reproduced insynchronization with the main part of the reference source, reproductionof the reference destination content is desirably stopped in an intervalin which the CMs of the reference source content are reproduced. In sucha case, transmitting device 100 may advance only the reference sourcetimeline and stop the reference destination timeline.

That is, transmitting device 100 updates time T_org in the CMreproduction period, and continues to set, to subsequent time T_ref, avalue of time T_ref which comes immediately before a generation intervalof the CMs. After the CM reproduction period ends, transmitting device100 updates time T_org and time T_ref together as usual. In this case,transmitting device 100 desirably transmits a TEMI access unitassociated with each PTS per AV access unit of the reference sourcestream. Further, transmitting device 100 may store, in the TEMI accessunit, identification information indicating that the referencedestination timeline or the reference destination timeline stops.Alternatively, transmitting device 100 may insert a non-reproductionperiod in the reference destination content in the CM reproductionperiod without stopping the reference destination timeline. For example,transmitting device 100 can generate a non-reproduction period by usinga function of “empty edit” in MP4. This method is applicable not only toCMs but also to a case where an interval which is not a target ofsynchronization and reproduction of the reference source and thereference destination is present.

Thus, the transmitting method according to the present exemplaryembodiment is a transmitting method for transmitting a first streamrelated to content of an image or audio, and includes transmitting afirst stream including, for example, timing update identificationinformation id1 such as is_timing_update illustrated in FIG. 2, thefirst time (corresponding to time T_org described above) and the secondtime (corresponding to time T_ref described above). Timing updateidentification information id1 indicates whether or not a correspondencerelationship between a first reference clock and a second referenceclock has been updated, the first reference clock being used to transmitand receive the first stream, and the second reference clock being usedto transmit and receive a second stream related to another content to bereproduced in synchronization with content related to the first stream.Further, the first time is a time of the first reference clock, and asecond time is a time of the second reference clock, the second timebeing associated with the first time based on the updated correspondencerelationship.

Thus, timing update identification information id1 is transmitted to thereceiving device. Consequently, the receiving device can easilydetermine whether or not the correspondence relationship between thefirst reference clock and the second reference clock has been updatedbased only on timing update identification information id1 withoutmaking determination by performing analysis using the first time and thesecond time. As a result, only when it is determined that thecorrespondence relationship has been updated, the receiving device needsto specify the updated correspondence relationship by using the firsttime and the second time and, consequently, a processing amount of thereceiving device can be suppressed. Consequently, it is possible toefficiently perform timeline extension.

Further, the transmitting method according to the present exemplaryembodiment includes determining whether or not the correspondencerelationship has been updated, and generating timing updateidentification information id1 indicating a result of the determination.Furthermore, the determination of whether or not the correspondencerelationship has been updated includes determining whether or not adiscontinuous change is present in a change in a time indicated by thefirst reference clock, and determining, when it is determined that thediscontinuous change is present, that the correspondence relationshiphas been updated, as shown in step S102 in FIG. 4. Still further, thefirst reference clock is, for example, a PCR according to MPEG2-TS.Moreover, the transmitting method according to the present exemplaryembodiment includes calculating an offset value (i.e., time offset)corresponding to the updated correspondence relationship between thefirst reference clock and the second reference clock, as shown in stepS104 in FIG. 4, when it is determined that the correspondencerelationship has been updated. Further, the first time and the secondtime that is specified based on the calculated offset value are includedin the first stream. More specifically, timing update identificationinformation id1 and the second time are stored in the TEMI access unitof the first stream, and the first time is stored in the header of thePES packet in which the TEMI access unit is stored.

FIG. 5 is a block diagram illustrating a configuration of the receivingdevice according to the present exemplary embodiment.

Receiving device 200 includes TEMI_AU obtaining unit 201, default statedetermining unit 202, update determining unit 203 and time synchronizer204.

TEMI_AU obtaining unit 201 obtains a TEMI access unit. Default statedetermining unit 202 determines whether or not synchronizing of bothtimelines has been finished before decoding of content starts. Updatedetermining unit 203 determines whether or not a correspondencerelationship between the both timelines has been updated. Timesynchronizer 204 performs time synchronization processing describedbelow on the both timelines.

FIG. 6 is a flowchart illustrating an example of a processing operationof receiving device 200 according to the present exemplary embodiment.

First, TEMI_AU obtaining unit 201 of receiving device 200 obtains a TEMIaccess unit (step S201). Next, before decoding of content is started,default state determining unit 202 determines whether or notsynchronizing of the reference source timeline and the referencedestination timeline has been finished (step S202). Note that, whendecoding of content is started, it is necessary to synchronize thereference source timeline and the reference destination timeline.

In step S202, when it is determined that the synchronization has beenfinished (“YES” in step S202), update determining unit 203 determineswhether or not timing update identification information id1(is_timing_update=1) indicating that the correspondence relationshipbetween the reference source timeline and the reference destinationtimeline has been updated has been set to the TEMI access unit (stepS203).

When it is determined in step S202 that the synchronization has not beenfinished (“NO” in step S202) or it is determined in step S203 thattiming update identification information id1 has been set (“YES” in stepS203), time synchronizer 204 performs time synchronization processing onthe reference source timeline and the reference destination timeline.

The time synchronization processing is to associate a time between thereference source timeline and the reference destination timeline. When,for example, time T_org of the reference source timeline matches withtime T_ref of the reference destination timeline, time synchronizer 204associates arbitrary time t1 of the reference source timeline and timet2 of the reference destination timeline according to the following(Equation 1).

t2=t1×(timescale2/timescale1)+(T_ref−(T_org×timescale2/timescale1))  (Equation 1)

Receiving device 200 synchronizes and reproduces an access unit of thereference source stream and an access unit of the reference destinationstream based on (Equation 1).

Not only when the correspondence between the timelines has been updatedbut also when the correspondence relationship has not been updated,receiving device 200 may perform time synchronization processing in stepS204. For example, receiving device 200 may cyclically perform the timesynchronization processing.

Further, when timing update identification information id1 is not storedin the TEMI access unit, receiving device 200 may determine whether ornot the reference clock has been updated according to another method,and may perform the time synchronization processing when it isdetermined that the reference clock has been updated.

For example, when the reference source stream is transmitted accordingto MPEG2-TS, receiving device 200 determines the update (discontinuity)of the reference clock based on a value of discontinuity_indicator of aTS packet in which a PCR is stored. Further, receiving device 200performs the time synchronization processing when it is determined thatthe discontinuity has occurred. Alternatively, receiving device 200 maycyclically obtain a TEMI access unit and perform the timesynchronization processing without making determination on the update ofthe reference clock.

Thus, the receiving method according to the present exemplary embodimentis a receiving method for receiving a first stream related to content ofan image or audio, and includes receiving a first stream including, forexample, timing update identification information id1 such asis_timing_update shown in FIG. 2, the first time (corresponding to timeT_org) and the second time (corresponding to time T_ref). Timing updateidentification information id1 indicates whether or not a correspondencerelationship between a first reference clock and a second referenceclock has been updated, the first reference clock being used to transmitand receive the first stream, and the second reference clock being usedto transmit and receive a second stream related to another content to bereproduced in synchronization with content related to the first stream.Further, the first time is a time of the first reference clock, and asecond time is a time of the second reference clock, the second timebeing associated with the first time based on the updated correspondencerelationship.

Thus, timing update identification information id1 is received byreceiving device 200. Consequently, receiving device 200 can easilydetermine whether or not the correspondence relationship between thefirst reference clock and the second reference clock has been updatedbased only on timing update identification information id1 withoutdeliberately making determination by performing analysis using the firsttime and the second time. As a result, only when it is determined thatthe correspondence relationship has been updated, the receiving deviceneeds to specify the updated correspondence relationship by using thefirst time and the second time and, consequently, a processing amountcan be suppressed. Consequently, it is possible to efficiently performtimeline extension.

That is, the receiving method according to the present exemplaryembodiment includes determining whether or not the correspondencerelationship has been updated based on timing update identificationinformation id1 included in the first stream, as shown in step S203 inFIG. 6. Further, when it is determined that the correspondencerelationship has been updated, the time synchronization processing isperformed as shown in step S204 in FIG. 6. That is, the updatedcorrespondence relationship is specified based on the first time and thesecond time included in the first stream. Further, content related tothe first stream and content related to the second stream aresynchronized and reproduced based on the updated correspondencerelationship.

Furthermore, when the updated correspondence relationship is specified,timescales of the first and second reference clocks are represented byTs1, Ts2, respectively, and the first and second times are representedby T_org, T_ref, respectively, and an arbitrary time of the firstreference clock is represented by t. In this case, the updatedcorrespondence relationship is specified by associating a time of thesecond reference clock calculated according tot×(Ts2/Ts1)+(T_ref−(T_org×Ts2/Ts1)) with the time t of the firstreference clock. Note that, Ts1, Ts2 correspond to timescale 1,timescale 2 in (Equation 1).

Modified Example 1

Modified examples of the present exemplary embodiment will be describedbelow.

Transmitting device 100 may describe access information (e.g. URL) inaddon_location of a TEMI access unit to obtain content managementinformation. The management information is, for example, MPD (MediaPresentation Description) according to MPEG-DASH or configurationinformation of a package (an asset which configures the package or a URLfor obtaining an asset) according to MMT.

When, for example, a reference destination stream is transmittedaccording to MPEG-DASH, receiving device 200 first obtains fromaddon_location of the TEMI access unit a URL for accessing an MPDaccording to MPEG-DASH. Then, receiving device 200 accesses this URL,obtains the MPD, and starts obtaining content of MPEG-DASH.

Furthermore, when a reference destination stream is transmittedaccording to an RTP, one encoded data of video or audio is generallytransmitted by using one RTP stream. Hence, transmitting device 100 maydescribe a URL for accessing content management information in which aplurality of RTP streams such as videos or audios are collected.

Alternatively, when a reference source stream is transmitted accordingto MPEG2-TS, transmitting device 100 may store management information ofreference destination content by using a descriptor of a PMT indicatinga PID of a stream of a TEMI access unit. In this case, transmittingdevice 100 may set a flag of has_location to 0, refer to the PMT andstore the management information of the reference destination contentwithout storing addon_location.

Transmitting device 100 may store management information as a descriptorrelated to a TEMI access unit, in a second loop of the PMT.Alternatively, transmitting device 100 may store information indicatingpresence or absence of reference destination content, in a descriptor ofa first loop of the PMT, and store management information of referencedestination content in the second loop when the reference destinationcontent is present.

Further, when reference source content and reference destination contenthave a dependence relationship, transmitting device 100 may storeinformation indicating the dependence relationship, in the PMTindicating information of a TEMI access unit. When reference sourcecontent is transmitted according to MPEG2-TS, the reference sourcecontent and a PID of a TEMI access unit are indicated by an identicalPMT. When, for example, reference destination content includes extensiondata of reference source content, it is possible to extend a time (e.g.frame rate), a space (e.g. resolution), and a bit depth by decodingextension data. More specifically, it is possible to increase the framerate from 60 fps to 120 fps or increase the resolution from 1920×1080pixels to 3840×2160 pixels. In such a case, the reference destinationcontent is extension data, and transmitting device 100 may store, in thePMT, information indicating what type of extension can be performed.Further, transmitting device 100 may store, in the PMT, informationindicating display positions of reference source content and referencedestination content.

Alternatively, when specific data of reference source content andreference destination content have a dependence relationship,transmitting device 100 may store, in the PMT, information indicatingwhich items of data have the dependence relationship. For example, thereference source content includes audio and video, and the referencedestination content includes caption data associated with the video ofthe reference source content. In this case, transmitting device 100 mayindicate that the video of the reference source content and thereference destination content are associated. That is, transmittingdevice 100 associates a PID of a TS packet in which video is stored, anda PID of a TS packet in which a TEMI access unit is stored.Alternatively, transmitting device 100 may indicate a PID of a TS packetin which the associated video is stored as relevant data in managementinformation of the reference destination content.

The information may be stored in the management information of thereference destination content obtained according to addon_location.

Receiving device 200 obtains content management information or adependence relationship between items of reference source content andreference destination content based on the above information, andreproduces these items of content.

Second Modified Example

FIGS. 7A to 7C are views each illustrating a syntax of a TEMI accessunit according to the present modified example.

The TEMI access unit includes a descriptor (temi_descriptor) asillustrated in FIG. 7A. This descriptor (temi_descriptor) includes atimeline descriptor (temi_timeline_descriptor) illustrated in FIG. 7B,and location descriptor (temi_location_descriptor) illustrated in FIG.7C.

The timeline descriptor illustrated in FIG. 7B is a descriptorindicating a correspondence relationship between both timelines, andincludes information related to a reference destination timeline.is_timing_update which is timing update identification information id1is stored in this timeline descriptor.

The location descriptor illustrated in FIG. 7C is a descriptorindicating information related to a location of reference destinationcontent.

That is, in a TEMI access unit, a correspondence relationship betweentimelines of content transmitted according to reference source MPEG2-TSand reference destination content, and location information of thereference destination content are described in different formats ofdescriptors.

A characteristic element of the location descriptor will be describedbelow. force_reload is a flag indicating whether entity data ofreference destination content or content management information needs tobe obtained again. This management information is content managementinformation such as an MPD according to MPEG-DASH which is referencedestination.

is_announcement is a flag indicating whether or not the referencedestination content can be obtained. When this flag is set to 1,timescale is indicated, and a time required until the referencedestination content can be obtained is indicated intime_before_activation.

addon_location is location information (e.g. URL) indicating a locationof the reference destination content.

splicing_flag is splicing identification information for instructingsplicing of items of content, and is a flag indicating reproduction andstop of the reference destination content. That is, when this flag isset to 1, reproduction of the reference destination content indicated ina descriptor to which this flag is not set is interrupted, and thereference destination content indicated in a descriptor to which thisflag is set is reproduced. For example, in location descriptor A,splicing_flag=0 is set, and reference destination content A isindicated. For example, in location descriptor B, splicing_flag=1 isset, and reference destination content B is indicated. In this case,when location descriptor B is received in a state where referencedestination content A is reproduced, receiving device 200 interrupts thereproduction of reference destination content A and reproduces referencedestination content B. Receiving device 200 resumes reproduction ofreference destination content A after finishing reproduction ofreference destination content B.

media_timestamp or npt corresponds to media_time_anchor described above,and indicates time T_ref in the reference destination timeline.

Next, a random access in a stream including the TEMI access unit(referred to as a TEMI stream below) will be described.

FIG. 8A is a view for describing a random access performed by receivingdevice 200. Receiving device 200 obtains and analyzes a locationdescriptor, and obtains and reproduces the reference destinationcontent. More specifically, as illustrated in FIG. 8A, receiving device200 obtains location descriptor A at time T0, and starts reproduction ofcontent A indicated by this location descriptor A. Subsequently,receiving device 200 obtains location descriptor B at time T1. Inlocation descriptor B, splicing_flag is set to 1. Hence, receivingdevice 200 interrupts reproduction of content A, and starts reproductionof content B indicated by location descriptor B. A splice interval whichis a reproduction period of content B ends at time T2. Hence, receivingdevice 200 resumes reproduction of content A after time T2.

Location descriptor A is cyclically transmitted from time T0, andlocation descriptor B is cyclically transmitted from time T1. Further, asplice interval is an interval in which content indicated by a locationdescriptor in which splicing_flag is set to 1 is reproduced.

In this case, receiving device 200 performs a random access. Forexample, receiving device 200 starts reception from time T at whichlocation descriptor B is transmitted. In this case, receiving device 200receives location descriptor B. As a result, receiving device 200 startsreproduction of content B. However, in this case, receiving device 200does not receive location descriptor A and therefore cannot specifycontent whose reproduction is to be resumed after the reproduction ofcontent B is finished.

Hence, in the present modified example, transmitting device 100transmits a location descriptor indicating content which is reproducedimmediately after the splice interval, at an end time of the spliceinterval, or before the end time. Thus, even when receiving device 200cannot obtain a location descriptor transmitted immediately before alocation descriptor in which splicing_flag has been set to 1, receivingdevice 200 can specify and reproduce source content whose reproductionis to be resumed after the splice interval ends.

In the example illustrated in FIG. 8A, transmitting device 100retransmits location descriptor A at time T2. When transmitting locationdescriptor A before the end time of the splice interval, transmittingdevice 100 may set a flag of is_announcement to 1, and indicate inlocation descriptor A that content A can be obtained from time T2.is_announcement is originally a flag indicating a time at which contentcan be obtained, but is_announcement may indicate a reproduction starttime of reference destination content whose reproduction is resumedafter the splice interval. An access unit whose reproduction is startedis determined based on a DTS (Decoding Time Stamp) or a PTS per accessunit in the reference destination content.

In this case, when the reference source timeline and the referencedestination timeline are different, receiving device 200 performs timesynchronization processing on both timelines based on synchronizationinformation of a timeline obtained from temi_timeline_descriptor (e.g.timescale and media_timestamp). Further, transmitting device 100 may seta flag of force_reload to 0 in a location descriptor to beretransmitted. Thus, when retransmitted location descriptor A isreceived, receiving device 200 which has already obtained locationdescriptor A can determine that it is not necessary to obtain thereference destination content again.

Transmitting device 100 may store, in a location descriptor to beretransmitted, information indicating content which is reproducedimmediately after a splice interval or information indicating contentswhose contents of a location descriptor have already been transmitted.Further, transmitting device 100 may indicate more significantinformation in a descriptor indicating an attribute of a TEMI stream.This significant information is information indicating that, even whenreception starts from an arbitrary TEMI access unit, referencedestination content indicated by a TEMI access unit whose reception hasbeen started, and reference destination content which temporally comeslater can be uniquely decided by retransmitting a location descriptor.

The example illustrated in FIG. 8A is an ideal example in whichreproduction of reference destination content can be started immediatelyafter a location descriptor is obtained. However, actually, untilcontent is obtained and starts being reproduced after a locationdescriptor is obtained, a time lag occurs depending on a bandwidth or acongestion situation of a communication network.

FIG. 8B is a view for describing processing operations of transmittingdevice 100 and receiving device 200 when a time lag is taken intoaccount.

When, for example, location descriptor A is obtained at time T0,receiving device 200 starts reproduction of content A at time S0.Further, transmitting device 100 may transmit a location descriptor bytaking into account a time lag which occurs until reproduction ofreference destination content is started after the location descriptoris obtained. That is, transmitting device 100 starts transmittingtemi_location_descriptor prior to a time corresponding to a DTS or a PTSof a head access unit of reference destination content. For example,when the DTS of the head access unit of the reference destinationcontent is converted into a value of a PCR according to reference sourceMPEG2-TS, the value takes 1000000 and a time lag is 600000. In thiscase, transmitting device 100 transmits temi_location_descriptor at orbefore a time at which the PCR value corresponds to 400000. Thus, it isassured that receiving device 200 reproduces reference destinationcontent from time 1000000.

Next, a random access point of a TEMI stream including the TEMI accessunit will be described.

Receiving device 200 needs to be able to obtain information related toreference destination content, and more specifically, needs to obtainthis information immediately after a reception starting operation ofselecting a channel for broadcasting or performing communication. Forexample, in a case of broadcasting, receiving device 200 obtains a PIDof a TEMI stream indicated by a PMT of a program to be received, andstarts reception of a TEMI stream. Hence, transmitting device 100desirably cyclically transmits a timeline descriptor and a locationdescriptor of a TEMI access unit similarly to a PMT of MPEG2-TS forbroadcasting.

A location descriptor only needs to be transmitted to receiving device200 when contents of the location descriptor are updated. This isbecause, analyzing a location descriptor of the same contents every timea TEMI access unit is received leads to an increase in a processing loadof receiving device 200.

A timeline descriptor only needs to be transmitted depending on afrequency at which timelines of reference source content and referencedestination content are synchronized by receiving device 200, andbasically only needs to be transmitted when discontinuity occurs in aPCR.

Hence, transmitting device 100 may signal to receiving device 200 thatthe contents of the location descriptor have been updated. For example,transmitting device 100 performs signaling by providing a counterindicating a version number to a location descriptor. Alternatively,transmitting device 100 performs signaling by settingrandom_access_indicator in a header of a TS packet. Further,transmitting device 100 may provide a new flag to a reserved areaimmediately after CRC_flag in a TEMI access unit. In this case, whenrandom_access_indicator is used, transmitting device 100 setsrandom_access_indicator to 1. More specifically, transmitting device 100sets random_access_indicator in a header of a TS packet including headdata of a TEMI access unit including the updated location descriptor.For example, a value of splicing_flag, a value of force_reload, orreference destination content indicated by a setting of new location_idis changed as contents of a location descriptor.

In this case, receiving device 200 obtains reference destination contentbased on contents of a TEMI access unit received immediately afterchannel selection for broadcasting, associates timelines of referencesource content and reference destination content, and startsreproduction of the reference destination content. After thereproduction starts, receiving device 200 determines whether or not thecontents of the location descriptor have been updated based on a valueof random_access_indicator, and switches the reference destinationcontent based on the updated contents in a case where the contents havebeen updated. Meanwhile, receiving device 200 monitors a value ofdiscontinuity_indicator of a header of a TS packet, and determineswhether or not discontinuity has occurred in a PCR. Receiving device 200refers to a timeline descriptor in the TEMI access unit, and tries againto associate (i.e., synchronize) both timelines when the discontinuityoccurs in the PCR. That is, a head location descriptor or a TEMI accessunit including a location descriptor which is transmitted first aftercontents are updated may be a random access point.

Further, the TEMI access unit includes only one of a timeline descriptorand a location descriptor and does not include a descriptor obtained byreceiving device 200 in some cases. In this case, receiving device 200continues to obtain a TEMI access unit until a TEMI access unit includedin an obtaining target descriptor can be received.

A random access point may be defined as follows.

As described with reference to the example in FIG. 8A, when reception ofa TEMI access unit in a splice interval is started, content which isreproduced immediately after the splice interval cannot be obtained insome cases. That is, when reception is started from an arbitrary TEMIaccess unit (AU(i)) and when information for obtaining content which isreproduced immediately after reference destination content indicated by(AU(i)) is indicated only in a TEMI access unit whose transmission ordercomes before (AU(i)), (AU(i)) does not become a random access point.Hence, when the information for obtaining content which is reproducedimmediately after the reference destination content indicated by (AU(i))is stored in a TEMI access unit whose transmission order comes after(AU(i)), this (AU(i)) may be a random access unit.

FIG. 9 is a flowchart illustrating an example of a processing operationof receiving device 200 according to the present modified example.

First, receiving device 200 analyzes a timeline descriptor and alocation descriptor of a TEMI access unit received immediately afterchannel selection. Thus, receiving device 200 specifies the referencedestination content, and obtains information necessary for associatingtimelines of the reference destination content and reference sourcecontent (step S301). Next, receiving device 200 receives the referencedestination content, and starts synchronizing and reproducing thereference destination content and the reference source content (stepS302).

Next, receiving device 200 determines whether or not contents of alocation descriptor have been updated (step S303). In this case, when itis determined that the contents have been updated (“YES” in step S303),receiving device 200 switches the reference destination content orchanges a URL of the reference destination content according to theupdated contents. Thus, receiving device 200 obtains the updatedreference destination content (step S304).

Next, receiving device 200 determines whether or not discontinuity of aPCR has occurred (step S305). In this case, when it is determined thatthe discontinuity has occurred (“YES” in step S305), receiving device200 refers to a timeline descriptor, and synchronizes again timelines ofthe time reference source content and the reference destination content(step S306).

Receiving device 200 may not execute steps S303, S304, or may notexecute steps S305, S306. That is, receiving device 200 may execute oneof processing including steps S303, S304 and processing including stepsS305, S306.

Thus, according to the transmitting method according to the presentexemplary embodiment, a TEMI access unit includes a timeline descriptorand a location descriptor. Further, the timeline descriptor storestiming update identification information id1 and a second time, and thelocation descriptor stores location information (e.g. URL) indicating alocation of content related to a second stream. Furthermore, thetransmitting method according to the present exemplary embodimentfurther includes transmitting another location descriptor includingsplicing identification information (e.g. splicing_flag) for stoppingreproduction of content related to the second stream and startingreproduction of content related to a third stream, and locationinformation (e.g., URL) indicating a location of the content related tothe third stream after the location descriptor is transmitted. Stillfurther, the location descriptor is retransmitted by a time when areproduction period of the content related to the third stream ends. Forexample, the location descriptor, the another location descriptor, thecontent related to the second stream, and the content related to thethird stream correspond to location descriptor A, location descriptor B,content A, and content B illustrated in FIGS. 8A and 8B, respectively.

Meanwhile, a receiving method according to the present exemplaryembodiment includes stopping reproduction of content related to thesecond stream and starting reproduction of content related to a thirdstream when a first descriptor including splicing identificationinformation (e.g. splicing_flag) instructing splicing of items ofcontent and location information indicating the content related to thethird stream are received. Further, reproduction of the content relatedto the second stream is resumed after a reproduction period ends when asecond descriptor including location information indicating a locationof the content related to the second stream is received by the time whenthe reproduction period of the content related to the third stream ends.For example, the first descriptor, the second descriptor, the contentrelated to the second stream, and the content related to the thirdstream correspond to location descriptor B, location descriptor A,content A, and content B illustrated in FIGS. 8A and 8B, respectively.

Thus, the another location descriptor is transmitted as the firstdescriptor to the receiving device. Consequently, receiving device 200can stop reproduction of the content related to the second stream basedon the splicing identification information stored in the anotherlocation descriptor, and reproduce the content related to the thirdstream. Further, the location descriptor which stores the locationinformation indicating a location of the content related to the secondstream is retransmitted as the second descriptor to receiving device 200by the time when a reproduction period of the content related to thethird stream ends. Consequently, even when reception of the first streamis started in this reproduction period, at the end of the reproductionperiod, receiving device 200 can easily specify the content to bereproduced next and reproduce the content. That is, similar to a casewhere reception of the first stream which is the reference source streamis started before this reproduction period, receiving device 200 canresume reproduction of the content related to the source second stream,the content which has been reproduced before the content related to thethird stream.

Third Modified Example

Hereinafter, a reproduction method in a splice interval will bedescribed.

Upon timeline extension, when temi_location_descriptor to whichsplicing_flag indicating 1 has been set is received, a receiving deviceinterrupts reproduction of content which has been reproduced so far.Further, the receiving device reproduces reference destination contentin which splicing_flag has been set to 1. However, when referencedestination content is transmitted according to MPEG-DASH, MMT, orMPEG2-TS, a DTS or a PTS of an access unit of reference destinationcontent is indicated in a stream such as MPEG-DASH or MMT. Hence, thereceiving device cannot uniquely decide an operation particularly whensplicing_flag is set.

Hence, according to the present modified example, the followingtransmitting method and receiving method can be used to uniquely decidean operation of reproducing reference destination content in a spliceinterval. In this case, a format which does not indicate an absolutevalue of a DTS or a PTS of an access unit as in MP4 will be referred toas a relative time type format. Further, a format which indicates anabsolute value of a DTS or a PTS of an access unit as in MPEG-DASH orMMT will be referred to as an absolute time type format.

According to the relative time type format, a DTS or a PTS indicated ina header of a PES packet in which a TEMI access unit includingtemi_location_descriptor indicating reference destination content isstored is handled as time information of a head access unit of thereference destination content. In this case, upon receipt oftemi_location_descriptor, receiving device 200 decides time informationbased on a header (a DTS or a PTS) of a PES packet in which acorresponding TEMI access unit is stored. Hence, for example, whenstarting reproduction of reference destination content from a time atwhich a PCR value takes 90000, and cyclically transmitting a TEMI accessunit, transmitting device 100 sets to 90000 a PTS value of a headerincluded in the PES packet.

Meanwhile, according to the absolute time type format, time informationof an access unit is indicated in a header in the format as a valuebased on a reference clock in each format. The reference clock is an NTPvalue according to MPEG-DASH or MMT, and is a PCR value according toMPEG2-TS. Receiving device 200 reproduces reference destination contentaccording to time information of an access unit. As described withreference to the example illustrated in FIG. 8B, a time lag occurs untilreference destination content can be reproduced aftertemi_location_descriptor is received. In this case, a DTS or a PTS maybe delayed according to a time lag. Further, since a timeline does notexist in the relative time type format, and thereforetemi_timeline_descriptor is not necessary.

Furthermore, in case where splicing_flag indicating 1 has been set, aDTS or a PTS may not be decided by the above method. That is, after areception time of temi_location_descriptor to which this splicing_flaghas been set, reproduction may be started at a point of time at whichreference destination content can be reproduced. Thus, flag informationindicating whether or not to reproduce reference destination contentbased on a reception time of temi_location_descriptor to whichsplicing_flag indicating 1 has been set may be stored in a descriptor.

Further, by setting a value of a field of nb_addons in a syntaxillustrated in FIG. 7C to 2 or more, it is possible to specify aplurality of items of reference destination content. Methods ofreproducing a plurality of items of reference destination contentspecified in this way include (1) a method for simultaneouslyreproducing a plurality of items of reference destination content and(2) a method for reproducing a plurality of items of referencedestination content one by one in storage order. Hence, identificationinformation indicating which one of methods (1) and (2) is used toperform reproduction may be stored in a descriptor. Further, a pluralityof items of reference destination content may be grouped, andidentification information indicating which one of the methods (1) and(2) is used to perform reproduction may be stored in a descriptor pergroup.

Fourth Modified Example

Hereinafter, attribute information of a TEMI stream will be described.

The following information related to reproduction control of referencesource content and reference destination content may be indicated asattribute information of a TEMI stream.

When reference destination content is obtained via a communicationnetwork, delay occurs until reproduction can be started after receptionis started depending on bandwidth or congestion of the network. Hence,to synchronize and reproduce reference source content and referencedestination content, it is necessary to buffer data of the referencesource content until the reference destination content is ready, anddelay reproduction of the reference source content.

Alternatively, when it is possible to specify a range of data obtainedfrom the reference destination content, receiving device 200 obtainseach access unit in order from an access unit including a subsequent PTSof the reference destination content by taking into account a timerequired to buffer the reference destination content. For example,receiving device 200 obtains each access unit in order from an accessunit of the reference destination content including a PTS which comes,for example, 10 seconds after a PTS of an access unit of the referencesource content whose reproduction is started. Thus, receiving device 200can synchronize and reproduce both items of reference source content andreference destination content after 10 seconds.

Alternatively, (1) when multiview reproduction is performed or (2)reference source content is a basic layer upon scalable coding andreference destination content is an extended layer, strictsynchronization and reproduction are requested. However, except forthese cases, it is also possible to independently reproduce items ofreference source content and reference destination content withoutsynchronizing the items of reference source content and referencedestination content. Hence, by indicating a reproduction mode whichsupports each situation as attribute information of a TEMI stream,receiving device 200 can appropriately reproduce the reference sourcecontent and the reference destination content according to thisreproduction mode.

Further, attribute information may indicate whether or nottemi_timeline_descriptor is included in a TEMI stream. For example,temi_location_descriptor is present and temi_timeline_descriptor is notpresent in some cases. In such a case, attribute information canindicate that reference destination content is present but it is notnecessary to synchronize and reproduce the reference destination contentand the reference source content. Alternatively, in the above case, theattribute information may indicate that timelines of the referencedestination content and the reference source content are identical.Further, these pieces of information may be indicated by items ofindependent identification information.

When temi_timeline_descriptor is present, attribute informationindicating a timeline type may be indicated. The timeline type includesthree types of, for example, an NTP (Network Time Protocol), a time codedefined according to IETF of RFC5484, and a time stamp unique to amedium. The NTP, the time code, and the time stamp unique to a mediumare indicated by fields of has_ntp, has_timecode, and has_timestamp oftemi_location_descriptor, respectively.

Further, a descriptor in which the attribute information is stored maybe defined, and this descriptor may be stored in a 2nd loop indicatingan attribute per elementary stream in a PMT.

Fifth Modified Example

A storage order of descriptors in a TEMI access unit may be defined inadvance. For example, as long as a timeline of reference destinationcontent does not change, a size of temi_timeline_descriptor is fixed.Hence, each descriptor is stored in a TEMI access unit in order oftemi_timeline_descriptor, and temi_location_descriptor. Thus, a startposition of temi_location_descriptor in the TEMI access unit is fixed.Hence, it is possible to omit processing of analyzing temi_descr_tag andtemi_descr_length of temi_timeline_descriptor and deciding a startposition of temi_location_descriptor, and reduce processing of receivingdevice 200.

Further, the start position of a field indicating information of atimeline in temi_location_descriptor is also fixed. Hence, by obtainingdata at a position to which a fixed byte offset is added from a head ofan access unit, it is possible to obtain information of the timelinewithout analyzing a descriptor. When it is not assured thattemi_timeline_descriptor is always present, receiving device 200analyzes a value of temi_descr_tag of a head descriptor of an accessunit, and determines whether or not temi_(—) location_descriptor ispresent. Note that, each descriptor which is stored in a TEMI accessunit and a flag indicating that a storage order of each descriptor isfixed may be included in a descriptor as attribute information of a TEMIstream.

As location information of reference destination content, a PID of atransport stream for broadcasting may also be specified in addition to aURL used for obtaining content via a communication network.Alternatively, when reference destination content is transmitted via abroadcast channel different from a broadcast channel of reference sourcecontent, an identification number of a transport stream may be furtherspecified.

The transmitting method and the receiving method according to one aspectof the present disclosure have been described above with reference tothe exemplary embodiment. However, the present disclosure is not limitedthereto.

FIG. 10A is a flowchart illustrating the transmitting method accordingto one aspect of the present disclosure.

This transmitting method is a transmitting method for transmitting afirst stream related to content of an image or audio, and includes stepS11. In this step S11, the first stream including timing updateidentification information, a first time, and a second time istransmitted. The timing update identification information indicateswhether or not a correspondence relationship between a first referenceclock and a second reference clock has been updated, the first referenceclock being used to transmit and receive the first stream and the secondreference clock being used to transmit and receive a second streamrelated to another content to be reproduced in synchronization withcontent related to the first stream. The first time is a time of thefirst reference clock. The second time is a time of the second referenceclock, the second time being associated with the first time based on theupdated correspondence relationship.

Even this transmitting method can provide the same effect as the effectof the transmitting method according to the above exemplary embodiment.

FIG. 10B is a block diagram of the transmitting device according to oneaspect of the present disclosure.

Transmitting device 10 is a transmitting device which transmits thefirst stream related to content of an image or audio, and includestransmitter 11. Transmitter 11 transmits the first stream including thetiming update identification information, the first time, and the secondtime. Even transmitting device 10 having such a configuration canprovide the same effect as the effect of transmitting device 100according to the above exemplary embodiment.

Further, the present disclosure may be a data generating method or adata generating device which generates the first stream. That is, thedata generating method or the data generating device according to thepresent disclosure is a method or a device which generates the firststream related to content of an image or audio, and generates the firststream including the timing update identification information, the firsttime, and the second time. Further, the data generating method accordingto the present disclosure may include any processing operation, otherthan transmission, included in the above exemplary embodiment and themodified examples of the exemplary embodiment. The data generatingdevice according to the present disclosure may include any component,other than a component which performs transmission, included in theexemplary embodiment and the modified examples of the exemplaryembodiment.

FIG. 11A is a flowchart illustrating the receiving method according toone aspect of the present disclosure.

This receiving method is a receiving method for receiving the firststream related to content of an image or audio, and includes step S21.In this step S21, the first stream including the timing updateidentification information, the first time, and the second time isreceived. The timing update identification information indicates whetheror not a correspondence relationship between a first reference clock anda second reference clock has been updated, the first reference clockbeing used to transmit and receive the first stream and the secondreference clock being used to transmit and receive a second streamrelated to another content to be reproduced in synchronization withcontent related to the first stream. The first time is a time of thefirst reference clock. The second time is a time of the second referenceclock, the second time being associated with the first time based on theupdated correspondence relationship.

Even this receiving method can provide the same effect as the effect ofthe receiving method according to the above exemplary embodiment.

FIG. 11B is a block diagram of the receiving device according to oneaspect of the present disclosure.

Receiving device 20 is a receiving device which receives the firststream related to content of an image or audio, and includes receiver21. Receiver 21 receives the first stream including the timing updateidentification information, the first time, and the second time. Evenreceiving device 20 having such a configuration can provide the sameeffect as the effect of receiving device 200 according to the aboveexemplary embodiment.

In the above exemplary embodiment and modified examples of the exemplaryembodiment, each component may be configured by dedicated hardware ormay be realized by executing a software program suitable to eachcomponent. Each component may be realized when a program executing unitsuch as a CPU or a processor reads and executes a software programrecorded in a hard disk or a recording medium such as a semiconductormemory. In this case, software which realizes the transmitting device orthe receiving device according to the above exemplary embodiment is thefollowing program. That is, this program causes a computer to executethe steps included in the flowchart illustrated in FIG. 10A or FIG. 11A.

Further, the following cases are included in the present disclosure.

(1) Each of the above devices is more specifically a computer systemincluding a microprocessor, a ROM, a RAM, a hard disk unit, a displayunit, a keyboard, and a mouse. In the RAM or the hard disk unit, acomputer program is stored. The microprocessor operates according to thecomputer program, so that each device achieves a function of eachdevice. The computer program described herein is configured by combininga plurality of command codes giving instructions to a computer toachieve a predetermined function.

(2) Part or the entirety of components which constitute each of theabove devices may be configured as one system LSI (Large ScaleIntegration). The system LSI is a super multifunction LSI manufacturedby integrating a plurality of components on one chip, and is morespecifically a computer system which is configured to include amicroprocessor, a ROM, and a RAM. In the RAM, the computer program isstored. The microprocessor operates according to the computer program,so that the system LSI achieves a function of the system LSI.

(3) Part or the entirety of components which constitute each of theabove devices may be configured as an IC card or a single module whichis detachable from each device. The IC card or the module is a computersystem which includes a microprocessor, a ROM, and a RAM. The IC card orthe module may include the above super multifunction LSI. Themicroprocessor operates according to the computer program, so that theIC card or the module achieves a function of the IC card or the module.This IC card or this module may be tamper resistant.

(4) The present disclosure may be the method described above. Further, acomputer program may cause the computer to realize these methods or adigital signal may include the computer program.

Furthermore, the present disclosure may cause the computer program orthe digital signal to be recorded in computer-readable recording mediasuch as a flexible disk, a hard disk, a CD-ROM, a MO, a DVD, a DVD-ROM,a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), and asemiconductor memory. Still further, the digital signal may be recordedin these recording media.

Moreover, according to the present disclosure, the computer program orthe digital signal may be transmitted via an electric communicationline, a wireless or wired communication line, a network which istypically the Internet, or data broadcasting.

Further, according to the present disclosure, a computer system mayinclude a microprocessor and a memory, the memory may store the computerprogram, and the microprocessor may operate according to the computerprogram.

Further, by recording the program or the digital signal in the recordingmedium and transmitting the program or the digital signal, or bytransporting the program or the digital signal via the network or thelike, the program or the digital signal may be executed by anotherindependent computer system.

(5) The exemplary embodiment and the modified examples may be combined.

The transmitting method and the receiving method according to one or aplurality of aspects have been described above based on the exemplaryembodiment and the modified examples of the exemplary embodiment.However, the present disclosure is not limited to the exemplaryembodiments. As long as it does not deviate from the gist of the presentdisclosure, exemplary embodiments provided by applying variousdeformations which a person skilled in the art conceives, to the presentexemplary embodiment, and exemplary embodiments constructed by combiningcomponents in different exemplary embodiments may be incorporated in ascope of one

The transmitting method and the receiving method according to thepresent disclosure provide an effect that timeline extension can beefficiently performed, and is applicable to, for example, a device orequipment which performs media transport on video data or audio data.

What is claimed is:
 1. A transmitting method for transmitting a firststream related to content of an image or audio, the method comprising:transmitting the first stream, the first stream including: timing updateidentification information indicating whether or not a correspondencerelationship between a first reference clock and a second referenceclock has been updated, the first reference clock being used to transmitand receive the first stream, and the second reference clock being usedto transmit and receive a second stream related to another content to bereproduced in synchronization with the content related to the firststream; a first time according to the first reference clock; and asecond time according to the second reference clock, the second timebeing associated with the first time based on the updated correspondencerelationship.
 2. The transmitting method according to claim 1, furthercomprising: determining whether or not the correspondence relationshiphas been updated; and generating the timing update identificationinformation indicating a result of the determination.
 3. Thetransmitting method according to claim 2, wherein the determination ofwhether or not the correspondence relationship has been updated includesdetermining whether or not a discontinuous change is present in afluctuation of a time indicated by the first reference clock, anddetermining, when it is determined that the discontinuous change ispresent, that the correspondence relationship has been updated.
 4. Thetransmitting method according to claim 3, wherein the first referenceclock is a PCR (Program Clock Reference) according to MPEG2-TS (MovingPicture Experts Group-2 Transport Stream).
 5. The transmitting methodaccording to claim 2, further comprising, when it is determined that thecorrespondence relationship has been updated: calculating an offsetvalue corresponding to the updated correspondence relationship betweenthe first reference clock and the second reference clock; and including,in the first stream, the first time and the second time that isspecified based on the calculated offset value.
 6. The transmittingmethod according to claim 1, further comprising storing the timingupdate identification information and the second time in a TEMI(Timeline and External Media Information) access unit of the firststream.
 7. The transmitting method according to claim 6, furthercomprising storing the first time in a header of a PES (PacketizedElementary Stream) packet in which the TEMI access unit is stored. 8.The transmitting method according to claim 6, wherein the TEMI accessunit includes a timeline descriptor and a location descriptor, and thetransmitting method further comprises: storing the timing updateidentification information and the second time in the timelinedescriptor; and storing location information in the location descriptor,the location information indicating a location of the content related tothe second stream.
 9. The transmitting method according to claim 8,further comprising: transmitting another location descriptor after thelocation descriptor is transmitted, the another location descriptorincluding splicing identification information and location information,the splicing identification information being used to stop reproductionof the content related to the second stream and start reproduction ofcontent related to a third stream, and the location informationindicating a location of the content related to the third stream; andretransmitting the location descriptor by a time when a reproductionperiod of the content related to the third stream ends.
 10. A receivingmethod for receiving a first stream related to content of an image oraudio, the method comprising: receiving the first stream, the firststream including: timing update identification information indicatingwhether or not a correspondence relationship between a first referenceclock and a second reference clock has been updated, the first referenceclock being used to transmit and receive the first stream, and thesecond reference clock being used to transmit and receive a secondstream related to another content to be reproduced in synchronizationwith the content related to the first stream; a first time according tothe first reference clock; and a second time according to the secondreference clock, the second time being associated with the first timebased on the updated correspondence relationship.
 11. The receivingmethod according to claim 10, further comprising: determining whether ornot the correspondence relationship has been updated, based on thetiming update identification information included in the first stream,the method further comprising, when it is determined that thecorrespondence relationship has been updated: specifying the updatedcorrespondence relationship based on the first time and the second timeincluded in the first stream; and synchronizing and reproducing contentrelated to the first stream and content related to the second streambased on the updated correspondence relationship.
 12. The receivingmethod according to claim 11, wherein the specifying of the updatedcorrespondence relationship includes, when timescales of the firstreference clock and the second reference clock are represented by Ts1and Ts2, respectively, and the first time and the second time arerepresented by T_org and T_ref, respectively, and an arbitrary time ofthe first reference clock is represented by t, specifying the updatedcorrespondence relationship by associating a time according to thesecond reference clock, which is calculated according tot×(Ts2/Ts1)+(T_ref−(T_org×Ts2/Ts1)), with the time t according to thefirst reference clock.
 13. The receiving method according to claim 10,further comprising: stopping reproduction of content related to thesecond stream and starting reproduction of content related to a thirdstream when a first descriptor including splicing identificationinformation and location information is received, the splicingidentification information instructing splicing of items of content, andthe location information indicating the content related to the thirdstream; and resuming reproduction of the content related to the secondstream after a reproduction period ends when a second descriptorincluding location information indicating a location of the contentrelated to the second stream is received by the time when thereproduction period of the content related to the third stream ends. 14.A transmitting device that transmits a first stream related to contentof an image or audio, the transmitting device comprising: a transmitterthat transmits the first stream, the first stream including timingupdate identification information indicating whether or not acorrespondence relationship between a first reference clock and a secondreference clock has been updated, the first reference clock being usedto transmit and receive the first stream, and the second reference clockbeing used to transmit and receive a second stream related to anothercontent to be reproduced in synchronization with the content related tothe first stream; a first time according to the first reference clock;and a second time according to the second reference clock, the secondtime being associated with the first time based on the updatedcorrespondence relationship.
 15. A receiving device that receives afirst stream related to content of an image or audio, the receivingdevice comprising: a receiver that receives the first stream, the firststream including: timing update identification information indicatingwhether or not a correspondence relationship between a first referenceclock and a second reference clock has been updated, the first referenceclock being used to transmit and receive the first stream, and thesecond reference clock being used to transmit and receive a secondstream related to another content to be reproduced in synchronizationwith the content related to the first stream; a first time according tothe first reference clock; and a second time according to the secondreference clock, the second time being associated with the first timebased on the updated correspondence relationship.